This project aims to adapt the existing Invisible Drum Set—originally implemented with an Arduino Nano and BNO055 IMU sensor—onto the E155 hardware platform, consisting of an STM32 MCU and UPduino FPGA. It will enable playing virtual drums through air gestures detected by the IMU, with real-time audio synthesis and low latency.
The data pipeline in our system relies on multiple communication protocols between the different hardware components. The BNO085 sensor communicates with the Arduino Nano ESP32 using the I²C protocol, with the high-level SHTP protocol layered on top to enable structured transfer of fused motion data. Once the sensor data is received by the ESP32, it is forwarded to the FPGA over an SPI interface, where the FPGA operates as the SPI slave and the ESP32 serves as the SPI master. On the FPGA, the data is processed and packaged into the custom transmission format used for the system.
From there, the FPGA sends the packaged data to the STM32 microcontroller through a second SPI link, again with the FPGA acting as the slave and the STM32 as the master. In the microcontroller, the data is further interpreted to identify which drum corresponds to the detected motion input. The resulting drum-selection signal is then output through a digital-to-analog converter (DAC) to drive a speaker that plays the correct drum sound.
This multi-protocol architecture enables reliable motion data transfer across all computing elements in the system.
Data Flow: BNO085 Sensor (I2C/SHTP) → Arduino ESP32 → [SPI] → FPGA → [SPI] → STM32 MCU → DAC → Speaker
The BNO085 is a 9-axis intelligent IMU that integrates an accelerometer, gyroscope, and magnetometer with an onboard sensor-fusion processor. It communicates through the Sensor Hub Transport Protocol (SHTP), a structured, packet-based protocol that runs over I²C, SPI, or UART.
Key Features: - Onboard ARM Cortex-M0+ microcontroller running Bosch’s BSX Lite fusion firmware - Performs sensor fusion (combining accel + gyro + mag) internally - Provides high-level motion data like quaternions and rotation vectors - SHTP organizes communication into channels with sequence numbers for reliable packet delivery
The Arduino Nano ESP32 combines the ESP32-S3’s processing power with the Arduino ecosystem, providing integrated Wi-Fi and Bluetooth connectivity. The system uses the Adafruit_BNO08x library, which abstracts low-level SHTP packet parsing and directly exposes orientation and motion data. Communication with the BNO085 uses I²C protocol with default address (0x4A).
Library: Adafruit_BNO08x
The FPGA serves as a data bridge between the Arduino ESP32 and STM32 MCU: - Receives sensor data from ESP32 via SPI (slave mode) - Processes and packages data into custom transmission format - Sends packaged data to STM32 MCU via SPI (slave mode) - Implements dual SPI slave interfaces
Key Modules: drum_trigger_top.sv, arduino_spi_slave.sv, spi_slave_mcu.sv
The STM32L432KC microcontroller receives gesture data from the FPGA via SPI, unpacks WAV files from flash memory (parsing RIFF headers and extracting audio samples), and plays the samples through the 12-bit DAC on PA4 at the correct sample rate to drive the speaker.
Validation using logic analyzer confirms correct data transfer. The packet begins with header byte 0xAA, followed by three 16-bit signed values for Euler angles (roll, pitch, yaw).
| Parameter | Raw Bytes (Hex) | Value (Decimal) | Interpretation |
|---|---|---|---|
| Header | AA | — | Packet start marker |
| Roll | 03 74 | 0x0374 = 884 | ~88.4° (divide by 10) |
| Pitch | FF 31 | 0xFF31 = –207 | ~–20.7° (divide by 10) |
| Yaw | 1F 0D | 0x1F0D = 7949 | ~794.9° (before wrap-around) |
The decoded values match Arduino serial monitor output, confirming SPI protocol is functioning correctly and data formatting matches the intended system design.
SPI communication between FPGA and MCU was validated using a logic analyzer, confirming correct data transfer and protocol compliance.
The 12-bit DAC on PA4 successfully outputs audio samples at the correct sample rate, enabling real-time drum sound playback through the speaker.
TODO: Add complete system block diagram and pin connection diagram
Please add: - System block diagram showing all components and communication protocols - Pin connection diagram with all interconnections, signal names, and pin numbers
Save diagrams to Images/ directory.
TODO: Add GitHub repository URL
View Complete Source Code on GitHub
The repository contains all source code, including FPGA Verilog modules, MCU C code, Arduino/ESP32 code, testbenches, and documentation files.
TODO: Add photos and video
Please add: - Photos of the complete system, hardware components, and wiring - Video demonstration showing the system in operation
Video should be hosted in the provided Google Drive folder and named: <LastName1><LastName2><VideoName> (e.g., BellidoStralkaInvisibleDrumSet)
| Material | Quantity | Cost | Total |
|---|---|---|---|
| BNO085 IMU Sensor | 2 | $24.95 | $49.90 |
| Speaker (8Ω, 4W) | 1 | Stockroom | $0.00 |
| Switches (drums) | 2 | Stockroom | $0.00 |
| Potentiometer | 1 | Stockroom | $0.00 |
| SD Card | 1 | Stockroom | $0.00 |
| TOTAL | $49.90 |
Total estimated cost under $50, excluding development hardware.
We want to thank staff engineer Xavier Walter and Professor Matthew Spencer for their constant support.